Skip to content

Keep the in-memory hosted engine off the disk for Go workspaces - #2031

Open
Mikola Lysenko (mikolalysenko) wants to merge 12 commits into
mainfrom
fix/hosted-memory-go-pathspace
Open

Mikola Lysenko (mikolalysenko) wants to merge 12 commits into
mainfrom
fix/hosted-memory-go-pathspace

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #2011 (fix/go-discovery-workspace), which has now landed on main as b69fc03; this PR targets main.

Problem

The hosted engine hands the golang rewriter the Go workspace it read (HostedGoWorkspace). In memory (ProjectView::Memory) its paths are root-relative with an empty root. The rewriter still judged them with the disk checks: replace_host, rooted_at, selection and host_conflict resolved paths against the process's working directory and probed <cwd>/go.mod. The in-memory engine's answer therefore depended on where the server process was started.

Example: an in-memory project with a root go.mod and a go.work that only has use ./b. If the cwd had no go.mod, the rewriter refused with "nowhere to go: … holds neither go.work nor a go.mod". If the cwd had one, it gave the correct "does not use this module" answer.

Fix

  • The workspace knows where its paths live. GoWorkspace now has files: Option<BTreeSet<PathBuf>>. In memory it lists the root-relative files that exist; on disk it is None. Every method goes through GoWorkspace::paths() (PathSpace::from_files). The _in(PathSpace, ..) variants from Read the governing go.work in Go discovery #2011 and their Disk-default wrappers are removed, so no caller can judge an in-memory workspace against the disk. The rewriter's code is unchanged from the base.
  • One in-memory builder. go_workspace::read_memory builds the in-memory workspace for both the hosted engine and discovery, through a small MemoryFiles reader trait. The engine reads through read_advisory; discovery uses its guarded reads, which diagnose an unparseable member go.mod. A file is in files when it exists, whether readable, a link, an entry without content, or a go.mod go cannot parse. Existence is checked without reading the file.
  • GoWorkspace::selection no longer shows a main-module member through the disk display_path.

I checked the rest of the in-memory engine's Go path: the memory branch of read_go_workspace reads only through the view, and go_build_list::select_in is pure. No other disk, cwd or env probes remain on that path.

Review findings

  1. files counted readable, not existing, files; engine and discovery built it differently. Confirmed, fixed. The engine skipped a symlinked or content-less root go.mod. Discovery skipped an unparseable one. Both now go through read_memory, which uses existence. The new tests hosted::engine::tests::memory_go_workspace_never_probes_the_cwd and vex::discover::golang::tests::in_memory_workspace_files_are_the_files_that_exist use the same three projects: a valid, an unparseable and a symlinked root go.mod, with use ./b. Both expect the same files and the "not a member" verdict. Under the old engine rule the symlinked case loses go.mod and fails. Note: in discovery the root go.mod's presence is only consulted for a parsed go.mod, so its old rule never changed a verdict; it is aligned anyway.
  2. members_within counted ../out as in-root in memory. Confirmed, fixed. Only all-Normal relative paths count as within the root, so absolute members are excluded too. memory_members_within_excludes_members_outside_the_root fails without the fix (["a", "../out", "/abs", ""]) and also checks the NoModule member hint.
  3. Test helper memory_go_files ignored a root go.mod outside the members. Fixed. go_workspace_key and memory_go_files now take root_go_mod. with_root_go_work passes whether the fixture has one.
  4. Store the PathSpace on the workspace. Done. See Fix above.
  5. Shared in-memory reader. Done. read_memory returns the workspace and its in-root members. Discovery derives its sum files from those members.
  6. PathSpace::from_files. Added, used by GoWorkspace::paths. Discovery's Workspace::paths and HostedGoWorkspace::paths are gone.
  7. Root go.mod read only for .is_some(). Fixed. It is now an existence check. The engine's check also counts unreadable entries.
  8. read_go_workspace pub(crate) only for a test. Fixed. The test moved into hosted::engine's tests and the function is private again.

Tests

  • cargo test -p socket-patch-core --lib -- hosted:: vex::discover:: vendor::go golang: 912 passed.
  • CLI --test hosted_memory_engine: 49 passed. --test hosted_memory_parity: 42 passed.
  • cargo clippy --locked --workspace --all-features -- -D warnings and cargo fmt --all -- --check are clean.

Brought current with #2011, plus round-2 review findings (cloud routine)

Merged the updated #2011 in. It merged cleanly. The core Go and hosted tests (657) and hosted_memory_parity (42) pass.

Review findings, round 2 (/code-review high on 2ff156a)

  1. The in-memory engine dropped an unparseable member go.mod: confirmed and fixed. The disk loader keeps that text. As a result, a member that declares the patched module (MainModule), or replaces it, was refused on disk but pinned over in memory. MemoryFiles::unparseable now says whether the text is kept: the engine keeps it, and discovery still drops it after its diagnostic. Test: memory_go_workspace_never_probes_the_cwd (extended). It fails with the engine dropping the text.

Declined:

  • Double normalize_for_read of the root go.mod. It only matters for pathological double-BOM or quote-nesting cases. Follow-up.
  • exists_no_follow counts an implied in-memory go.mod/ directory. It only changes which refusal is named.
  • Duplicate probes, the read_sum trait method, the duplicated member reader and the private doc link. Cleanup only.

Tests (cloud run): cargo test -p socket-patch-core --lib -- vex::discover:: utils::go hosted:: vendor::go gives 837 passed, golden included. CLI --test hosted_memory_parity gives 42 passed. clippy and fmt are clean.

Main merged after #2011 landed (burn-down agent)

df219a66 merges main at b69fc03. main's vendor/go_workspace.rs and vex/discover/golang.rs are byte-identical to the #2011 tip (ec87efaf) this branch had already merged, so both conflicts resolve to this branch's side. cargo test -p socket-patch-core --lib -- vex::discover:: utils::go hosted:: vendor::go: 841 passed; cargo check -p socket-patch-core --all-targets clean.


Note

Medium Risk
Changes Go workspace conflict and replace-host decisions for in-memory projects; behavior is aligned with disk semantics but touches patch redirect and discovery paths.

Overview
In-memory golang patching previously used disk path checks (including probing the process cwd for go.mod), so workspace conflict answers could differ depending on where the server started. This PR makes the in-memory workspace self-contained.

GoWorkspace now carries an optional files set listing root-relative paths that exist in memory. PathSpace::from_files and GoWorkspace::paths() route all conflict, replace-host, and member logic through lexical comparisons when files is set, removing the *_in(PathSpace, …) APIs so callers cannot mix disk and memory semantics.

A shared go_workspace::read_memory builder (via the MemoryFiles trait) loads in-memory workspaces for both the hosted engine (AdvisoryGoFiles) and discovery (ViewGoFiles), counting existence (links, unreadable entries, unparseable go.mod) rather than only readable content. The engine still retains unparseable member go.mod text for conflict weighing, matching disk behavior.

members_within in memory now excludes paths with .. or absolute components. replace_host treats an existing but unreadable root go.mod as a real root module. Tests assert engine and discovery agree and that memory never probes cwd.

Reviewed by Cursor Bugbot for commit 04a1a9d. Configure here.


Generated by Claude Code

Go discovery read only a go.work at the project root and only members
inside the root. With GOWORK unset it ignored an ancestor go.work and read
the root go.mod in module mode, so in a member of a parent workspace it
could attest a go.mod replace that the workspace overrides, doesn't `use`,
or makes inert by version selection. Hosted and vendored modes resolve the
workspace go itself would use and refuse to write such a replace.

On disk, discovery now loads the governing workspace with the same
go_workspace::load_with loader hosted mode uses: GOWORK (environment, then
the go env -w file), else the nearest go.work at or above the root, with
`use` members anywhere. Out-of-root members' requires and replaces now
count toward selection and conflicts. In-root members are still re-read
through the view, so recognition and parse diagnostics stay the same.

A wired Socket replace that this workspace keeps from the build, at
exactly the points where hosted mode's host_conflict check refuses, stays a
ref (list, rollback and remove still find it) but is marked unattested
with the new UnattestedKind::GoWorkspaceOverride. The same applies to a
replace in the root go.work while GOWORK makes go build another one.
vex_sources reports these under the existing `go_workspace_override`
reason, the one vex's own go.mod gate already uses. A replace that
workspace selection makes inert is still diagnosed, as in module mode.
A GOWORK that go rejects still withholds every ref as patched_ref_invalid.

In memory there is no directory to search from and no environment the
project builds under. The root go.work and its in-root members are the
workspace there, as the in-memory hosted engine reads them, and the same
conflict gate applies. GOWORK keeps its old effect in memory and GOFLAGS
is not read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Version-selection inertness in replaced_purl, and the hosted selection
checks, ran before Workspace::bypass. A Socket replace that the
governing go.work also refuses (not used, overridden by a go.work
replace, member conflict) was dropped as inert with a diagnostic
instead of being kept as a GoWorkspaceOverride ref, so list, rollback
and remove could not find wiring that re-running scan or vendor would
never clear. Compute the bypass first and skip the inertness checks for
a bypassed replace; it stays a ref and is reported unattested.

Found by Cursor Bugbot on #2011.

Co-Authored-By: Claude <noreply@anthropic.com>
This fixes the confirmed /code-review high findings on #2011. Each fix
has a test that fails on the previous commit.

- A Go workspace mark now applies to one file only. The new
  UnattestedKind::file_scoped is true only for GoWorkspaceOverride.
  vex_sources drops the bypassed ref from the candidate's basis and
  vetoes the patch only when no live ref of it remains. wiring_conflicts
  ignores bypassed refs. So a root go.work replace that go never reads no
  longer withholds the root go.mod's mirror of it. A leftover member pin
  that Socket's own go.work replace overrides no longer turns into a
  wiring conflict that also withholds the go.work patch, which is what
  vex.rs's WorkHosted exemption intends.
- A replace of a module that is itself a `use` member is diagnosed as
  inert again, both on disk and in memory, hosted and vendored. This
  check now runs before the workspace bypass.
- SumPins now also reads the sum files the loader already read: the
  governing go.work.sum and each out-of-root member's go.sum. They are
  used only when they lie inside the git repository.
- In memory, workspace conflicts are evaluated in
  go_workspace::PathSpace::Memory. Paths are compared lexically against
  the view's files and never against the filesystem or the working
  directory. conflict/work_conflict delegate to the new *_in variants,
  so disk behavior is unchanged.
- The snapshot probe list is now built by one shared helper
  (workspace_probes), spelled the way resolve_with spells it.
- A root go.work that go doesn't read is no longer judged unparseable,
  and its wiring stays an unattested ref.
- The go_workspace_override omission phrase now covers every case.
- read_loaded_workspace uses members_within. The view re-reads only the
  in-root members' go.mod, which recognition and parse diagnostics need.
  The duplicate go.sum and go.work.sum reads are gone.
- Tests that use a nested project now create a .git, so they keep
  working once reads are bounded to the repository root (#2025).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hosted engine hands the golang rewriter the Go workspace it read
(HostedGoWorkspace). In memory its paths are root-relative, but the
rewriter still judged them with the disk checks: replace_host,
rooted_at, selection and host_conflict canonicalized against the
process's cwd and probed `<cwd>/go.mod`. So an in-memory project whose
root module go.work does not `use` got "nowhere to go" (no go.mod)
when the server's cwd had none, and the member answer when it did.

HostedGoWorkspace now carries `files`: in memory, the root-relative
files that exist (go.work, a readable root go.mod, each in-root
member's readable go.mod), filled by the engine as discovery lists
them. The rewriter routes every workspace check through
`PathSpace::Memory` when it is set, via new `_in` variants of
`replace_host`, `selection`, `host_conflict` and `conflict` on
GoWorkspace / Loaded. Disk behaviour is unchanged (the old names
delegate with `PathSpace::Disk`).

`GoWorkspace::selection` also displayed a main-module member through
the disk display_path; discovery's build-list graph now carries the
workspace's PathSpace so in-memory discovery uses `selection_in` too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2984629 dropped every workspace-bypassed ref from wiring_conflicts, so a
leftover member pin that Socket's go.work replace overrides would not
withhold the go.work patch. That also caught the root go.mod. A
GOWORK=off build from the root reads the root go.mod, so a different
patch there still disagrees with go.work about what the build links.
The behavior is the same reasoning ReplaceHost::mirrors_root_go_mod
uses. e2e_vex_lockfile golang_go_mod_and_go_work_disagreeing_attest_neither
went red in coverage (2): it got go_workspace_override plus
go_version_not_selected skips, not one wiring_conflict skip.

Count bypassed refs in the root go.mod as wiring again. Member go.mod
pins are still exempt, as a_bypassed_leftover_pin_is_no_wiring_conflict
requires.

Co-Authored-By: Claude <noreply@anthropic.com>
- GoWorkspace carries its own `files` (in memory: the root-relative
  files that exist; `None` on disk) and every method routes through
  `GoWorkspace::paths()`. The `_in(PathSpace, ..)` variants and their
  Disk-default wrappers are gone, so no caller can judge an in-memory
  workspace against the disk or the cwd. HostedGoWorkspace::files and
  its paths() go with them; the rewriter's code is back to the base's.
- One builder for the in-memory workspace, go_workspace::read_memory,
  over a MemoryFiles reader: the hosted engine reads through
  read_advisory, discovery through its guarded reads (diagnosing an
  unparseable member go.mod). Both list a file in `files` when it
  exists (a link, an entry without content, a go.mod go cannot parse
  all count), checked without reading it, and both drop an unparseable
  member go.mod's text.
- members_within no longer counts `../out` or an absolute member as
  inside the empty in-memory root (strip_prefix("") matched them), so
  the NoModule hint names in-root members only.
- PathSpace::from_files builds the space from an optional file list.
- The cwd test moved to hosted::engine's tests; read_go_workspace is
  private again. The redirect fixtures list the root go.mod whenever
  the project has one, as read_memory does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Second review round on Go workspace discovery.

- A Socket replace of a module that is itself a `use` member (every
  replace of it is inert, and the writers refuse it with
  `WorkspaceConflict::MainModule`, which a re-run never clears) was
  dropped with a diagnostic, so rollback, remove and list lost wiring
  still in the file. It stays a ref, marked `GoWorkspaceOverride`
  with the same "is inert" detail (owner decision 4).
- go.sum / member sums / go.work.sum were read only when a hosted ref
  existed, so a sum left by a reverted patch was no longer recognized
  and a dead ledger claim could be re-derived from its raw text
  (rule 11). They are recognized either way.
- The plan both attested a purl (the live go.work patch) and reported
  it skipped for a lockfile-only candidate whose only ref is a stale,
  bypassed leftover pin of another patch. That candidate is now dropped
  with a note when another candidate attests the purl.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Second review round on the in-memory Go workspace.

The in-memory hosted engine dropped the text of a member go.mod go
cannot parse, while the disk loader keeps it: a member declaring the
patched module (every replace of it inert, `MainModule`) or replacing
it was refused on disk but pinned over in memory. `MemoryFiles::
unparseable` now says whether the text is kept: the engine keeps it,
discovery still drops it after its diagnostic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from fix/go-discovery-workspace to main November 7, 2026 00:55
#2011 squash-merged as b69fc03. Its go_workspace.rs and
discover/golang.rs on main equal the #2011 tip this branch already
merged (ec87efa), so both conflicts resolve to this branch's side.

Co-Authored-By: Claude <noreply@anthropic.com>
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

In memory, GoWorkspace::files counts a root go.mod that exists but
has no text the rewriter holds (a link, an unreadable file), and
conflict() reports such a root as "not a member" when go.work does not
use it. The rewriter passes has_go_mod from its candidate texts, so
replace_host saw no root module, chose go.work, and ran work_conflict:
the replace went into go.work, where it patches the members but never
the project. When the root is not a member, treat a root go.mod that
the in-memory workspace lists as the root's module, as on disk, so
go.mod hosts the replace and conflict() refuses it.

Found by Bugbot on df219a6.

Co-Authored-By: Claude <noreply@anthropic.com>
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 04a1a9d. Configure here.

@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review November 7, 2026 07:28
@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Nov 7, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Burn-down agent: took this out of draft and labeled it Ready for review at head 04a1a9d.

  • CI: 32/32 checks that ran are green on 04a1a9d after leaving draft; the other 50 were skipped by path filters. No main-wide failures.
  • Bugbot: reviewed 04a1a9d with no new findings. Its earlier finding 4500107857 (replace_host keeping go.mod for an in-memory root module that go.work doesn't use) was fixed in 04a1a9d and the thread is resolved.
  • Mergeable, and the PR doesn't touch CHANGELOG.md.
  • For the reviewer: GoWorkspace now carries files (an in-memory PathSpace), and the Disk-default _in(..) wrappers from Read the governing go.work in Go discovery #2011 are gone, so every caller goes through GoWorkspace::paths().

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does. The in-memory hosted engine judged Go workspaces with disk checks, so its answer depended on the server process's working directory (it probed <cwd>/go.mod). Now GoWorkspace carries the set of in-memory files that exist, and every check goes through GoWorkspace::paths(), so in memory paths are compared lexically and the disk is never touched. The engine and discovery now build the in-memory workspace with one shared builder (read_memory).

Risk: low to medium. Disk behavior is unchanged: files is None on disk and PathSpace::Disk uses the same same_dir_key as before. The changes only affect in-memory Go workspaces, where they fix a real nondeterminism. The _in(PathSpace, ..) APIs are removed, so no caller can mix the two modes.

Look here

Verified

  • Read the full diff (4 files, +627/−158). The PR description matches the code. No debug leftovers or unrelated edits, and CHANGELOG.md is untouched.
  • The in-memory reads stay inside the root: in_root_dir filters out .. and absolute use dirs before any read.
  • Ran cargo test -p socket-patch-core --lib -- vex::discover::golang vendor::go hosted::engine::tests::memory_go locally on 04a1a9d: 225 passed.
  • CI: 32/32 checks that ran are green (50 skipped by path filters). ci-ok and clippy are green. Bugbot is clean on 04a1a9d, and all review threads are resolved.

Changes I made: none.

Open questions: none blocking. The author declined some cleanups in the description: a double normalize_for_read of the root go.mod, and an implied in-memory go.mod/ directory that only changes which refusal is named. Both look fine to leave as follow-ups.

Auto-merge is armed: approving sends this PR straight to the merge queue.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants